Simulating Local Biases in Visual Attention in Neurocognitive Performance.

Typical visual perception includes an attention bias toward right hemisphere mediated global, holistic cortical processing. An atypically local, detail-oriented focus of attention is characteristic of left hemisphere processing and is often observed in patients whose field of attention is restricted by certain types of neurocognitive impairment. We designed the present pair of studies to induce a local attentional focus to observe its consequences on neurocognitive measures of visuospatial processing. In Experiment I, participants wore glasses mimicking simultanagnosia, a disorder of visual attention, to induce a narrowed, atypical attentional style while they completed visual neuropsychological tasks. This simulation impaired participants' capacities to visually synthesize and efficiently reproduce Complex Figure stimuli as measured with the Boston Qualitative Scoring System (BQSS), and it induced an atypical attentional style on Rorschach Performance Assessment System (R-PAS) responses. In Experiment II, participants wore glasses designed to provoke differential hemispheric activation, also hypothesized to influence style of visual attention; but this manipulation did not influence neurocognitive task performance. We discuss implications for the interpretation of BQSS and R-PAS scores and offer directions for future research.

New Versions of the MMPI and Rorschach: How Have Training Programs Responded?

In this brief article, we update the training of newer versions of the Minnesota Multiphasic Personality Inventory (MMPI) and Rorschach and compare to a 2015 assessment training survey of American Psychological Association accredited clinical psychology doctoral programs. The survey sample sizes for 2015, 2021, and 2022 were 83, 81, and 88, respectively. By 2015, of the programs teaching any adult MMPI version, almost all (94%) were still teaching the MMPI-2, and 68% had started teaching the MMPI-2-RF. In 2021 and 2022, respectively, almost all programs (96% and 94%) had started teaching the MMPI-2-RF or MMPI-3, although most were still teaching the MMPI-2 (77% and 66%). By 2015, of the programs teaching the Rorschach, 85% were still teaching the Comprehensive System (CS) and 60% had started teaching the Rorschach Performance Assessment System (R-PAS). In 2021 and 2022, respectively, most programs had started teaching R-PAS (77% and 77%) although many (65% and 50%) were still teaching the CS. Therefore, doctoral programs are indeed switching to newer versions of the MMPI and Rorschach, although more slowly than one might expect. We recommend that APA provide more guidance in selecting test versions for training programs, practitioners, and researchers.

Can the Rorschach be Administered Remotely? A Review of Options and a Pilot Study Using a Newly Developed R-PAS App.

The ongoing COVID-19 pandemic has required psychologists to adopt measures like physical distancing and mask wearing, though other safety procedures such as travel restrictions or prohibitions on in-person practice and research have fostered the use of tele-health tools. In this article, we review options for using the Rorschach task via videoconference and provide preliminary data from using a new electronic app for remote R-PAS administration to determine whether the remote administration in an electronic form yields different information than in-person administration with the cards in hand. As a pilot study, our focus is on the "first factor" of all Rorschach scores, i.e., complexity. Data were collected from 60 adult Italian community volunteers, and statistical analyses evaluated the extent to which the average complexity score significantly departed from R-PAS normative expectations (SS = 100), accompanied by Bayesian likelihoods for supporting the null hypothesis. Results suggest that the general level of complexity shown by the test-takers when administered the Rorschach remotely with the new R-PAS app closely resembles that previously observed using "standard" in-person procedures. Tentative analyses of other R-PAS scores suggested normative departures that could be due to the effects of the app, testing at home, or responses to the pandemic. We offer recommendations for future research and discuss practical implications.

Education and training guidelines for psychological assessment in health service psychology.

While recent survey findings suggest graduate programs in health service psychology (HSP) are allocating the same or increased time to education and training in psychological assessment over the last two decades, there is a lack of clear guidance for programs to implement practices associated with quality education and training. These Guidelines (found in full at https://www.apa.org/about/policy/guidelines-assessment-health-service.pdf) were developed to address this critical need. Developed by a task force of the American Psychological Association Board of Educational Affairs in 2018 and 2019, the Guidelines serve to inform faculty/supervisors, students, and the public as to quality practices associated with graduate education and training in psychological assessment. They are organized around seven domains: theory; psychological assessment process; psychometrics; tests and methods; ethics, legal issues, and professionalism; diversity; and supervision. These domains are drawn from a review of the scholarly literature on psychological assessment, as well as graduate psychology education and training. The domains and their associated Guidelines are interdependent, and, while some overlap exists among them, they should be considered in their entirety. While a summary of each section is provided in the present article, the full explanation of each domain is presented in the actual Guidelines document. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Correspondence of Maximum and Typical Performance Measures of Cognitive Processing.

We examined associations among cognitive processing measures that varied on a continuum from maximum to typical performance tasks in the context of an ego depletion study. Our intent was to replicate and extend the findings of Charek, Meyer, and Mihura, which showed that ego depletion had an expected effect on selected scores from the Rorschach inkblot task. We hypothesized that Rorschach variables indicative of cognitive sophistication would correlate with neuropsychological measures of cognitive ability and that Rorschach variables theoretically unassociated with cognitive processing would not correlate with those criterion measures. These hypotheses were supported, providing evidence of convergent and discriminant validity. We also hypothesized ego depletion effects on all measures; however, none were evident. Methodological considerations and implications of the findings are discussed.

The Effect of CS Administration or an R-Optimized Alternative on Potential Projective Material in Rorschach Responses From Six Studies and a Meta-Analysis of Their Findings.

The Rorschach Performance Assessment System (R-PAS; Meyer, Viglione, Mihura, Erard, & Erdberg, 2011) introduced R-optimized administration to reduce variability in the number of Responses (R). We provide new data from six studies of participants randomly assigned to receive a version of this method or Comprehensive System (CS; Exner, 2003) administration. We examine how administration methods affect 3 types of codes most likely to contain potential projective material and the frequency of these codes for the 1st, 2nd, 3rd, 4th, or last response to a card (R in Card). In a meta-analytic summary, we found 37% of responses have 1 type of code, 19% have 2 types, and 3% have all 3 types, with stable proportions across responses within cards. Importantly, administration method had no impact on potential projective variable means. Differential skew across samples made variability harder to interpret. Initial results suggesting differences in 3 of the 18 specific Type by R in Card pairs did not follow a coherent pattern and disappeared when using raw counts from all participants. Overall, data do not support concerns that R-optimized administration might alter potential projective processes, or make potentially "signature" last responses to the card any different in R-PAS than the CS.

The Effect of CS Administration or an R-Optimized Alternative on R-PAS Variables: A Meta-Analysis of Findings From Six Studies.

Controlling the number of Rorschach responses (R) as a method to reduce variability in the length of records has stimulated controversy among researchers for many years. Recently, the Rorschach Performance Assessment System (R-PAS; Meyer, Viglione, Mihura, Erard, & Erdberg, 2011 ) introduced an R-Optimized method to reduce variability in R. Using 4 published and 2 previously unpublished studies (N = 713), we examine the extent to which 51 Comprehensive System-based scores on the R-PAS profile pages are affected as a result of receiving Comprehensive System (CS; Exner, 2003 ) administration versus a version of R-Optimized administration. As hypothesized, R-the intended target of R-Optimized administration-showed reliable weighted average differences across each method of administration. As expected, its mean modestly increased and its standard deviation notably decreased. Also as hypothesized, the next largest effects were decreases in the variability (SD) of 2 variables directly related to R, R8910% and Complexity. No other reliable differences were observed. Therefore, because R-Optimized administration does not notably modify the existing CS-based normative values for other profiled R-PAS variables, the data do not support concerns that R-Optimized administration notably modifies the Rorschach task or that existing CS research data would not generalize to R-PAS. However, because R-Optimized administration reduces variability in R, it allows a single set of norms to apply readily to more people.

Why We Need a Formal Systematic Approach to Validating Psychological Tests: The Case of the Rorschach Comprehensive System.

This article documents and discusses the importance of using a formal systematic approach to validating psychological tests. To illustrate, results are presented from a systematic review of the validity findings cited in the Rorschach Comprehensive System (CS; Exner, 2003) test manual, originally conducted during the manuscript review process for Mihura, Meyer, Dumitrascu, and Bombel's (2013) CS meta-analyses. Our review documents (a) the degree to which the CS test manual reports validity findings for each test variable, (b) whether these findings are publicly accessible or unpublished studies coordinated by the test developer, and (c) the presence and nature of data discrepancies between the CS test manual and the cited source. Implications are discussed for the CS in particular, the Rorschach more generally, and psychological tests more broadly. Notably, a history of intensive scrutiny of the Rorschach has resulted in more stringent standards applied to it, even though its scales have more published and supportive construct validity meta-analyses than any other psychological test. Calls are made for (a) a mechanism to correct data errors in the scientific literature, (b) guidelines for test developers' key unpublished studies, and (c) systematic reviews and meta-analyses to become standard practice for all psychological tests.

Development and preliminary validation of a brief behavioral measure of psychotic propensity.

Extensive research demonstrates that the dimensional assessment of psychotic manifestations is a primary strength of the Rorschach inkblot task, which provides an in vivo sample of problem-solving behavior and normative standards concerning the logic and coherence of thought processes and the typicality of perceptual representations. This article presents foundational research for the Thought and Perception Assessment System (TPAS), a Rorschach-based system designed to assess solely for disordered thinking and perceptual aberrations using either the standard 10-card set of inkblots or alternative 3-, 4-, and 5-card short forms. Using data from three patient samples (ns = 61, 93, and 133) and one nonpatient sample (n = 118), we document the equivalence of mean scores across the full and short-form card sets. We also document satisfactory interrater reliability and validity for the full and short forms, as well as strong part-whole reliability coefficients between the short forms and the full form. Consistent with psychometric theory and the principle of aggregation, each type of coefficient decreased as a function of decreasing the number of cards. We discuss implications and future applications in research and clinical assessment.

The Incremental Validity of Self-Report and Performance-Based Methods for Assessing Hostility to Predict Cardiovascular Disease in Physicians.

We evaluated the utility of an integrative, multimethod approach for assessing hostility-related constructs to predict premature cardiovascular disease (CVD) and premature coronary heart disease (CHD) using participants from the Johns Hopkins Precursors Study, which was designed to identify risk factors for heart disease. Participants were assessed at baseline while in medical school from 1946 to 1962 (M age = 24.6) and have been followed annually since then. Baseline assessment included individually administered Rorschach protocols (N = 416) scored for aggressive imagery (i.e., Aggressive Content, Aggressive Past) and self-reports of 3 possible anger responses to stress. Cox regression analyses predicting morbidity or mortality by age 55 revealed a significant interaction effect; high levels of Aggressive Content with high self-reported hostility predicted an increased rate of premature CVD and CHD, and incrementally predicted the rate of these events after controlling for the significant covariates of smoking (CVD and CHD) and cholesterol (CHD) that were also assessed at baseline. The hostility and anger measures, as well as other baseline covariates, were not predictors of CVD risk factors assessed at midlife during follow-up. Overall, this integrative model of hostility illustrates the potential value of multimethod assessment to areas of health psychology and preventive medicine.

The Centrality of the Response Process in Construct Validity: An Illustration via the Rorschach Space Response.

Recently, psychologists have emphasized the response process-that is, the psychological operations and behaviors that lead to test scores-when designing psychological tests, interpreting their results, and refining their validity. To illustrate the centrality of the response process in construct validity and test interpretation, we provide a historical, conceptual, and empirical review of the main uses of the background white space of the Rorschach cards, called space reversal (SR) and space integration (SI) in the Rorschach Performance Assessment System. We show how SR and SI's unique response processes result in different interpretations, and that reviewing their literatures with these distinct interpretations in mind produces the expected patterns of convergent and discriminant validity. That is, SR was uniquely related to measures of oppositionality; SI was uniquely related to measures of cognitive complexity; and both SR and SI were related to measures of creativity. Our review further suggests that the Comprehensive System use of a single space code for all uses of white space likely led to its lack of meta-analytic support as a measure of oppositionality (Mihura, Meyer, Dumitrascu, & Bombel, 2013 ). We close by discussing the use of the response process to improve test interpretation, develop better measures, and advance the design of research.

Psychological Assessment Training in Clinical Psychology Doctoral Programs.

We surveyed American Psychological Association-accredited clinical psychology doctoral programs' (n = 83) training in psychological assessment-specifically, their coverage of various assessment topics and tests in courses and practica, and whether the training was optional or required. We report results overall and separately per training model (clinical science, scientist-practitioner, and practitioner-focused). Overall, our results suggest that psychological assessment training is as active, or even more active, than in previous years. Areas of increased emphasis include clinical interviewing and psychometrics; multimethod, outcomes, health, and collaborative or therapeutic assessment; and different types of cognitive and self-report personality tests. All or almost all practice-focused programs offered training with the Thematic Apperception Test and Rorschach compared to about half of the scientist-practitioner programs and a third of the clinical science programs. Although almost all programs reported teaching multimethod assessment, what constitutes different methods of assessing psychopathology should be clarified in future studies because many programs appear to rely on one method-self-report (especially clinical science programs). Although doctoral programs covered many assessment topics and tests in didactic courses, there appears to be a shortage of program-run opportunities for students to obtain applied assessment training. Finally, we encourage doctoral programs to be familiar with (a) internships' assessment expectations and opportunities, (b) the professional guidelines for assessment training, and

On Conducting Construct Validity Meta-Analyses for the Rorschach: A Reply to Tibon Czopp and Zeligman (2016).

We respond to Tibon Czopp and Zeligman's (2016) critique of our systematic reviews and meta-analyses of 65 Rorschach Comprehensive System (CS) variables published in Psychological Bulletin (2013). The authors endorsed our supportive findings but critiqued the same methodology when used for the 13 unsupported variables. Unfortunately, their commentary was based on significant misunderstandings of our meta-analytic method and results, such as thinking we used introspectively assessed criteria in classifying levels of support and reporting only a subset of our externally assessed criteria. We systematically address their arguments that our construct label and criterion variable choices were inaccurate and, therefore, meta-analytic validity for these 13 CS variables was artificially low. For example, the authors created new construct labels for these variables that they called "the customary CS interpretation," but did not describe their methodology nor provide evidence that their labels would result in better validity than ours. They cite studies they believe we should have included; we explain how these studies did not fit our inclusion criteria and that including them would have actually reduced the relevant CS variables' meta-analytic validity. Ultimately, criticisms alone cannot change meta-analytic support from negative to positive; Tibon Czopp and Zeligman would need to conduct their own construct validity meta-analyses.

The Impact of an Ego Depletion Manipulation on Performance-Based and Self-Report Assessment Measures.

We investigated the impact of ego depletion on selected Rorschach cognitive processing variables and self-reported affect states. Research indicates acts of effortful self-regulation transiently deplete a finite pool of cognitive resources, impairing performance on subsequent tasks requiring self-regulation. We predicted that relative to controls, ego-depleted participants' Rorschach protocols would have more spontaneous reactivity to color, less cognitive sophistication, and more frequent logical lapses in visualization, whereas self-reports would reflect greater fatigue and less attentiveness. The hypotheses were partially supported; despite a surprising absence of self-reported differences, ego-depleted participants had Rorschach protocols with lower scores on two variables indicative of sophisticated combinatory thinking, as well as higher levels of color receptivity; they also had lower scores on a composite variable computed across all hypothesized markers of complexity. In addition, self-reported achievement striving moderated the effect of the experimental manipulation on color receptivity, and in the Depletion condition it was associated with greater attentiveness to the tasks, more color reactivity, and less global synthetic processing. Results are discussed with an emphasis on the response process, methodological limitations and strengths, implications for calculating refined Rorschach scores, and the value of using multiple methods in research and experimental paradigms to validate assessment measures.

Standards, accuracy, and questions of bias in Rorschach meta-analyses: reply to Wood, Garb, Nezworski, Lilienfeld, and Duke (2015).

Wood, Garb, Nezworski, Lilienfeld, and Duke (2015) found our systematic review and meta-analyses of 65 Rorschach variables to be accurate and unbiased, and hence removed their previous recommendation for a moratorium on the applied use of the Rorschach. However, Wood et al. (2015) hypothesized that publication bias would exist for 4 Rorschach variables. To test this hypothesis, they replicated our meta-analyses for these 4 variables and added unpublished dissertations to the pool of articles. In the process, they used procedures that contradicted their standards and recommendations for sound Rorschach research, which consistently led to significantly lower effect sizes. In reviewing their meta-analyses, we found numerous methodological errors, data errors, and omitted studies. In contrast to their strict requirements for interrater reliability in the Rorschach meta-analyses of other researchers, they did not report interrater reliability for any of their coding and classification decisions. In addition, many of their conclusions were based on a narrative review of individual studies and post hoc analyses rather than their meta-analytic findings. Finally, we challenge their sole use of dissertations to test publication bias because (a) they failed to reconcile their conclusion that publication bias was present with the analyses we conducted showing its absence, and (b) we found numerous problems with dissertation study quality. In short, one cannot rely on the findings or the conclusions reported in Wood et al.

The association of gender, ethnicity, age, and education with Rorschach scores.

We examined the association of gender, ethnicity, age, and education with 60 Rorschach scores using three clinical and nonclinical samples of adults and youths (ns = 640, 249, and 241). As anticipated for our data sets, there were no reliable associations for gender, ethnicity, or adult age. However, in adults years of education was associated with variables indicative of complexity, the articulation of subtlety and nuance, cognitive synthesis, and coping resources. In the clinical sample of youths, increasing age was primarily associated with more conventional perception and less illogical thought processes. Limitations are discussed in conjunction with further research that could address them, along with implications for applied practice.

A meta-analysis of an implicit measure of personality functioning: the Mutuality of Autonomy Scale.

The Mutuality of Autonomy scale (MA) is a Rorschach variable designed to capture the degree to which individuals mentally represent self and other as mutually autonomous versus pathologically destructive (Urist, 1977). Discussions of the MA's validity found in articles and chapters usually claim good support, which we evaluated by a systematic review and meta-analysis of its construct validity. Overall, in a random effects analysis across 24 samples (N = 1,801) and 91 effect sizes, the MA scale was found to maintain a relationship of r =.20, 95% CI [.16,.25], with relevant validity criteria. We hypothesized that MA summary scores that aggregate more MA response-level data would maintain the strongest relationship with relevant validity criteria. Results supported this hypothesis (aggregated scoring method: r =.24, k = 57, S = 24; nonaggregated scoring methods: r =.15, k = 34, S = 10; p =.039, 2-tailed). Across 7 exploratory moderator analyses, only 1 (criterion method) produced significant results. Criteria derived from the Thematic Apperception Test produced smaller effects than clinician ratings, diagnostic differentiation, and self-attributed characteristics; criteria derived from observer reports produced smaller effects than clinician ratings and self-attributed characteristics. Implications of the study's findings are discussed in terms of both research and clinical work.

Rorschach scores in applied clinical practice: a survey of perceived validity by experienced clinicians.

We surveyed practicing clinicians who regularly used the Rorschach about the perceived clinical validity of specific Rorschach scores from many coding systems. The survey included quantitative feedback on the validity of specific variables as well as qualitative input in several areas, including the validity of specific variables, the potentially unique information that can be obtained from them, coding challenges associated with Comprehensive System (CS) codes, and recommendations for CS developments. Participants were recruited by applying a snowball sampling strategy. Based on responses from 246 experienced clinicians from 26 countries, composite judgments on rated variables were quite reliable (e.g., M α = .95 across 88 CS variables), despite limited agreement among any 2 judges. The aggregated judgments clearly differentiated among scores that were considered more and less clinically valid and the overall results aligned with recently obtained meta-analytic conclusions from the traditional validity literature (Mihura, Meyer, Dumitrascu, & Bombel, 2012). The judges also provided guidance concerning revisions and enhancements that would facilitate Rorschach-based assessment in the future. We discuss the implication of the quantitative and qualitative findings and provide suggestions for future directions based on the results.

The validity of individual Rorschach variables: systematic reviews and meta-analyses of the comprehensive system.

We systematically evaluated the peer-reviewed Rorschach validity literature for the 65 main variables in the popular Comprehensive System (CS). Across 53 meta-analyses examining variables against externally assessed criteria (e.g., observer ratings, psychiatric diagnosis), the mean validity was r = .27 (k = 770) as compared to r = .08 (k = 386) across 42 meta-analyses examining variables against introspectively assessed criteria (e.g., self-report). Using Hemphill's (2003) data-driven guidelines for interpreting the magnitude of assessment effect sizes with only externally assessed criteria, we found 13 variables had excellent support (r ≥ .33, p < .001; [Symbol: see text] FSN > 50), 17 had good support (r ≥ .21, p < .05, FSN ≥ 10), 10 had modest support (p < .05 and either r ≥ .21, FSN < 10, or r = .15-.20, FSN ≥ 10), 13 had little (p < .05 and either r = < .15 or FSN < 10) or no support (p > .05), and 12 had no construct-relevant validity studies. The variables with the strongest support were largely those that assess cognitive and perceptual processes (e.g., Perceptual-Thinking Index, Synthesized Response); those with the least support tended to be very rare (e.g., Color Projection) or some of the more recently developed scales (e.g., Egocentricity Index, Isolation Index). Our findings are less positive, more nuanced, and more inclusive than those reported in the CS test manual. We discuss study limitations and the implications for research and clinical practice, including the importance of using different methods in order to improve our understanding of people.

A principal components analysis of Rorschach aggression and hostility variables.

We examined the structure of 9 Rorschach variables related to hostility and aggression (Aggressive Movement, Morbid, Primary Process Aggression, Secondary Process Aggression, Aggressive Content, Aggressive Past, Strong Hostility, Lesser Hostility) in a sample of medical students (N= 225) from the Johns Hopkins Precursors Study (The Johns Hopkins University, 1999). Principal components analysis revealed 2 dimensions accounting for 58% of the total variance. These dimensions extended previous findings for a 2-component model of Rorschach aggressive imagery that had been identified using just 5 or 6 marker variables (Baity & Hilsenroth, 1999; Liebman, Porcerelli, & Abell, 2005). In light of this evidence, we draw an empirical link between the historical research literature and current studies of Rorschach aggression and hostility that helps organize their findings. We also offer suggestions for condensing the array of aggression-related measures to simplify Rorschach aggression scoring.